ConformalPrediction.jlPredictive Uncertainty Quantification in Machine Learning
Delft University of Technology
April 25, 2025
ConformalPrediction.jl (10min)Code along 💻 tinyurl.com/cpjcon2023
Conformal Prediction involves turning heuristic measures of Predictive Uncertainty into rigorous ones.
A first crucial step towards building trustworthy AI systems is to be transparent about predictive uncertainty.
Figure 1: Softmax output for class 1 (top left); non-conformity scores for calibration set (top right); (1-α)-quantile (bottom left); non-conformity function applied to test point (bottom right). Solid bars make it into prediction set.
ConformalPrediction.jlMLJ.jl.Flux.jl through MLJFlux.jl.Remember SymbolicRegression.jl by Miles Cranmer?
A simple MNIST classifier.
Ensemble Batch Prediction Intervals (Xu and Xie 2021) contributed by Mojtaba Farmanbar 📣.
Tutorial
How to Conformalize a Time Series Model (docs)
LaplaceRedux.jlContribute
LaplaceRedux.jl interfaced to MLJFlux.jl. Planning to add both ideas [#64].
CounterfactualExplanations.jl.Contribute
Currently working on full conformal training implementation [#62].
JuliaTaija collects Julia packages geared towards Trustworthy AI:
Contributions welcome! 😊